[Feature] Add scalar quantization support for vector extension#28
Merged
Merged
Conversation
adsharma
self-requested a review
July 13, 2026 19:44
adsharma
reviewed
Jul 13, 2026
adsharma
left a comment
Contributor
There was a problem hiding this comment.
Looks great! Couple of minor comments.
Tests should be green once the main repo's commit is in.
Persist SQ8 and SQ16 embeddings in an internal _QEMB node table so quantized index state participates in Ladybug's normal MVCC, WAL, and checkpoint lifecycle. Use the durable quantized rows for HNSW construction and on-disk search, with batched table-backed reads when caching is disabled and an aligned dense cache when caching is enabled. Preserve optional full-precision reranking and normalize cosine vectors for the quantized distance path. Keep the shared dense cache transactionally safe by tracking dirty writer transactions and invalidating it from commit and rollback callbacks. Add coverage for quantized index behavior and document storage layout, query flow, cache semantics, and benchmark results.
rahul-iyer
force-pushed
the
quantized-rerank-cache
branch
from
July 17, 2026 05:10
c0b7d47 to
907df8f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds scalar quantization support for HNSW vector indexes behind the quantization option.
This PR supports:
Dot product with quantization is rejected for now because the quantized metric path currently supports
L2/L2SQ/Cosine-style scoring.
Behavior
Without quantization, HNSW continues to use the existing full-precision float embedding path.
With quantization enabled:
Benchmark
Ran it on OpenAI 50k small
Results:
Speedup vs baseline:
Storage overhead vs baseline:
Correctness Notes
This PR also fixes several issues found during testing:
Quantized cache invalidation is currently coarse-grained: mutations invalidate the full quantized cache.
A future improvement should make this transaction-aware and row/version based.